Random variables

Figure File Name Link Description
01_basic_projection 01_basic_projection TeX PDF PNG SVG The projection of a random variable \(Y\) onto the line spanned by a random varibale \(X\).
01_corr_def 01_corr_def TeX PDF PNG SVG Geometric representation of random variables.
01_law_of_iterated_expectations 01_law_of_iterated_expectations TeX PDF PNG SVG The law of iterated expectations. Equivalence of the two-step projecttion and direct projection of \(Y\) onto \(\mathbf{1}\).
01_mse_decomposition 01_mse_decomposition TeX PDF PNG SVG Decomposition of mean squred error into the variance and the bias squared \(\left(a = \sqrt{\lVert\hat\theta - E(\hat\theta)\rVert^2} \right.\), \(b = \sqrt{\lVert \theta - E(\hat\theta) \rVert^2}\), \(\left. c = \sqrt{\lVert \hat\theta - \theta \rVert^2} \right)\).
01_pythagorean_theorem 01_pythagorean_theorem TeX PDF PNG SVG The Pythagorean theorem for random variables \(X\) and \(Y\).

Regression

Figure File Name Link Description
02_averages_final 02_averages_final TeX PDF PNG SVG Regression of \(y\) and \(\hat y\) on \(\mathbf{1}\).
02_averages_yhat_decomposed 02_averages_yhat_decomposed TeX PDF PNG SVG Regression of \(y\) on \(Lin(\mathbf{1},x)\) and decomposition of \(\hat y\) into a sum of \(\hat\beta_1 \mathbf{1}\) and \(\hat \beta_2 x\).
02_basic_projection 02_basic_projection TeX PDF PNG SVG Vector \(y\) projected onto vector \(x\).
02_correlation_constant_centered_variables 02_correlation_constant_centered_variables TeX PDF PNG SVG Centred vectors \(x^c\) and \(y^c\).
02_correlation_constant_proof 02_correlation_constant_proof TeX PDF PNG SVG Proof of \(sCorr(x + \alpha \mathbf{1}, y) = sCorr(x,y)\).
02_cramers_rule 02_cramers_rule TeX PDF PNG SVG OLS formula illustrated in \(\mathbb{R}^k\).
02_detremination_coefficient 02_detremination_coefficient TeX PDF PNG SVG Determination coefficient as squared \(\cos \varphi\) where \(a\) stands for \(\sqrt{RSS}\), \(b\) — \(\sqrt{TSS}\), \(c\) — \(\sqrt{ESS}\).
02_duality_final 02_duality_final TeX PDF PNG SVG New residuals translated to the origin of the unit circle.
02_duality_first_residuals_translated 02_duality_first_residuals_translated TeX PDF PNG SVG Residuals translated to the orgin of the unit circle.
02_duality_inversion 02_duality_inversion TeX PDF PNG SVG Example of inversion for vector \(a\).
02_duality_new_regressors 02_duality_new_regressors TeX PDF PNG SVG Regressors \(v_1\), \(v_2\) obtained from inversion of the residuals \(\hat{u}_1\), \(\hat{u}_2\).
02_duality_new_residuals 02_duality_new_residuals TeX PDF PNG SVG Regressions of \(v_1\) onto \(v_2\) and of \(v_2\) onto \(v_1\).
02_duality_original_regressors 02_duality_original_regressors TeX PDF PNG SVG Original regressors in unit circle.
02_fwl_v1_cleansed_variables 02_fwl_v1_cleansed_variables TeX PDF PNG SVG FWL: regression of \(y\) on \(z\) and of \(x\) on \(z\).
02_fwl_v1_final 02_fwl_v1_final TeX PDF PNG SVG FWL: point A stands for the origin, B — \(\hat\gamma z\), C — \(x\), D — \(\hat\alpha z\), E — intersection of vector \(x\) and line parallel to \(\tilde x\), F — \(\hat\beta_1^{(1)} x\), G — \(\hat\beta_1^{(2)} \tilde{x}\).
02_fwl_v1_final_lin 02_fwl_v1_final_lin TeX PDF PNG SVG FWL: \(Lin(x,z)\).
02_fwl_v1_translation 02_fwl_v1_translation TeX PDF PNG SVG FWL: translation of \(\tilde{x}\).
02_fwl_v1_yhat_decomposed 02_fwl_v1_yhat_decomposed TeX PDF PNG SVG FWL: regression of \(y\) on \(Lin(x,z)\).
02_fwl_v1_yhat_decomposed_lin 02_fwl_v1_yhat_decomposed_lin TeX PDF PNG SVG FWL: \(Lin(x, z)\).
02_fwl_v2_cleansed_regression 02_fwl_v2_cleansed_regression TeX PDF PNG SVG FWL: “cleansed” \(\tilde y\) regressed on “cleansed” \(\tilde{x}\).
02_fwl_v2_cleansed_variables 02_fwl_v2_cleansed_variables TeX PDF PNG SVG FWL: “cleansed” variables \(\tilde x\) and \(\tilde y\).
02_fwl_v2_final 02_fwl_v2_final TeX PDF PNG SVG Alternative proof for the Frisch-Waugh-Lovell theorem.
02_fwl_v2_similar_triangles 02_fwl_v2_similar_triangles TeX PDF PNG SVG FWL: similar triangles \(\bigtriangleup ABC \sim \bigtriangleup EDC\).
02_gmt 02_gmt TeX PDF PNG SVG Gauss-Markov theorem for the case of three regressors.
02_instr 02_instr TeX PDF PNG SVG Geometry of instrumental variables. \(A\) stands for \(\hat \beta_{IV} \hat x\), \(B\) — \(\hat x\), \(C\) — \(x\), \(D\) — \(\hat \beta_{IV} x\).
02_proxy 02_proxy TeX PDF PNG SVG Geometry of proxy variables.
02_rss_ess_tss_final 02_rss_ess_tss_final TeX PDF PNG SVG Illustration of the equality \((\sqrt{RSS})^2 + (\sqrt{ESS})^2 = (\sqrt{TSS})^2\) where \(a\) stands for \(\sqrt{RSS}\), \(b\) — \(\sqrt{TSS}\), \(c\) — \(\sqrt{ESS}\).
02_rss_ess_tss_sqr_ess 02_rss_ess_tss_sqr_ess TeX PDF PNG SVG Illustartion of ESS.
02_rss_ess_tss_sqr_rss 02_rss_ess_tss_sqr_rss TeX PDF PNG SVG Illustration of RSS.
02_rss_ess_tss_sqr_tss_ess 02_rss_ess_tss_sqr_tss_ess TeX PDF PNG SVG Total sum of squares and residual sum of squares.
02_rss_ess_tss_yhat 02_rss_ess_tss_yhat TeX PDF PNG SVG Residual sum of squares.
02_simple_regression_coefficient_basic 02_simple_regression_coefficient_basic TeX PDF PNG SVG Starting pictutre for regression illustrations.
02_simple_regression_coefficient_centred_variables 02_simple_regression_coefficient_centred_variables TeX PDF PNG SVG “Centred” \(x\) and \(y\), i.e., projected onto \(Lin^{\perp}(\mathbf{1})\).
02_simple_regression_coefficient_negative 02_simple_regression_coefficient_negative TeX PDF PNG SVG Proof of \(Corr(y, \hat y) = sign(\hat\beta_2)Corr(y, x)\) for the case of \(\beta_2 < 0\).
02_simple_regression_coefficient_yhat_projected 02_simple_regression_coefficient_yhat_projected TeX PDF PNG SVG “Centred” \(\hat y\), i.e., projected onto \(Lin^{\perp}(\mathbf{1})\).

Partial correlation

Figure File Name Link Description
03_partial_correlation_definition 03_partial_correlation_definition TeX PDF PNG SVG Partial correlation between \(X\) and \(Y\) while \(Z\) is fixed.
03_partial_correlation_regression_definition 03_partial_correlation_regression_definition TeX PDF PNG SVG Alternative definition of the partial correlation through regressions on two variables.
03_partial_correlation_regression_definition_lin 03_partial_correlation_regression_definition_lin TeX PDF PNG SVG Partial correlation definition through geometric mean / regression residuals, \(Lin^{\perp}(Z)\).
03_partial_correlation_residuals_x.svg 03_partial_correlation_residuals_x TeX PDF PNG SVG Residuals \(\hat u\) form regression of x onto \(Y\) and \(Z\), \(\hat u\) projected.
03_partial_correlation_residuals_xy 03_partial_correlation_residuals_xy TeX PDF PNG SVG The residuals of both regressions.
03_partial_correlation_residuals_xy_lin 03_partial_correlation_residuals_xy_lin TeX PDF PNG SVG Partial correlation definition through residuals, \(Lin^{\perp}(Z)\).
03_partial_correlation_residuals_y 03_partial_correlation_residuals_y TeX PDF PNG SVG Residuals \(\hat v\) from regression of \(Y\) onto \(X\) and \(Z\), \(\hat v\) projected.

Probability distributions

Figure File Name Link Description
04_chi_squared_example 04_chi_squared_example TeX PDF PNG SVG A \(3\)-dimensional example for the chi-squared distribution.
04_f_dist_example 04_f_dist_example TeX PDF PNG SVG F-distribution as the ratio of the projection lengths squared adjusted to the dimensions of the subspaces.
04_ftest 04_ftest TeX PDF PNG SVG F-statistic as the cotangent squared of \(\varphi\) where \(a\) stands for \(\sqrt{RSS_{UR}}\), \(b\) — \(\sqrt{RSS_{R} -RSS_{UR}}\), \(c\) — \(\sqrt{RSS_{R}}\).
04_normal 04_normal TeX PDF PNG SVG Black dots represent the gas molecules. The red dot stands for the one we catch. Its speed along the horizontal axis is \(v_1\), i.e., the first component of the velocity vector, and its speed along the vertical axis is \(v_2\).
04_ttest 04_ttest TeX PDF PNG SVG T-statistic as cotangent of \(\varphi\). Regression of \(y\) onto \(Lin(x, \mathbf{1})\) and appropriate projections.
04_ttest_lin 04_ttest_lin TeX PDF PNG SVG T-statistic as cotangent of \(\varphi\), \(Lin^{\perp}(\mathbf{1})\).